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Abstract 


Early  evaluation  of  the  architecture  of  a  system  or  a  product  line  of  systems  is  a  low-cost  risk 
reduction  method  for  determining  whether  the  system(s)  will  achieve  its  business  and  quality 
goals.  The  Architecture  Tradeoff  Analysis  Method  (ATAM)  is  an  architecture  evaluation  tech¬ 
nique  currently  evolving  at  the  Software  Engineering  Institute  (SEI).  The  input  to  the  ATAM 
consists  of  a  system  or  product  line  architecture  and  the  perspectives  of  stakeholders  involved 
with  that  system  or  product  line.  The  output  of  the  ATAM  is  (1)  a  collection  of  scenarios  that 
help  specify  the  context  of  the  system’s  or  product  line’s  use  and  the  product  line’s  evolution, 
(2)  improved  architectural  documentation  (usually),  and  (3)  analysis  results  (in  particular,  a  set 
of  issues  to  consider,  risks,  and  potential  sensitivity  and  tradeoff  points  within  the  architecture). 
Currently,  there  are  no  generally  accepted  industry-wide  standards  for  describing  a  system  ar¬ 
chitecture,  and  ATAM  evaluations  are  often  tailored  to  the  available  documentation. 

The  Architectures  Directorate  of  the  C4I  Integration  Support  Activity  (CISA),  Office  of  the  As¬ 
sistant  Secretary  of  Defense  for  Command,  Control,  Communications,  and  Intellegence 
(OASD[C3I])  has  defined  a  framework  for  architecture  development,  presentation,  and  integra¬ 
tion  to  be  used  across  the  military  services  and  defense  agencies.  This  framework  for  Com¬ 
mand,  Control,  Communications,  Computers,  Intelligence,  Surveillance,  and  Reconnaissance 
(C4ISR)  is  becoming  the  required  method  for  describing  information  systems  within  the  De¬ 
partment  of  Defense  (DoD)  and  other  U.S.  Government  agencies.  This  report  describes  how 
various  C4ISR  products  can  be  used  in  the  context  of  an  ATAM  evaluation  and  their  relative 
value  for  generating  quality  attribute-specific  scenarios  required  for  an  ATAM  evaluation. 
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1  The  Architecture  Tradeoff  Analysis 
Method  (ATAM) 


The  Architecture  Tradeoff  Analysis  Method  (ATAM)  is  an  architecture  evaluation  technique 
currently  evolving  at  the  Software  Engineering  Institute  (SEI).  The  input  to  the  ATAM  con¬ 
sists  of  a  system  or  product  line  architecture  and  the  perspectives  of  stakeholders  involved 
with  that  system  or  product  line.  The  ATAM  relies  on  generating  use-case  and  change-case 
scenarios  to  assess  the  architecture.  Use  cases  and  change  cases  are  described  below: 

•  Use  cases  represent  operational  conditions  that  the  architecture  should  support  and  dem¬ 
onstrate  the  effectiveness  of  the  architecture  to  satisfy  these  operating  conditions. 

•  Change  cases  represent  expected  system  modifications  that  may  cause  architectural 
changes  and  demonstrate  how  efficiently  the  architecture  can  be  modified. 

The  ATAM  achieves  its  architecture  evaluation  for  quality-attribute  goals  by  using  an  under¬ 
standing  of  the  architectural  mechanisms  that  are  used  to  achieve  particular  quality  goals  and 
the  implications  of  those  mechanisms.  Each  quality  attribute  is  examined  from  the  point  of 
view  of  three  perspectives:  what  external  stimuli  causes  the  architecture  to  respond  or  change, 
what  mechanisms  are  used  within  the  architecture  to  control  the  response,  and  how  the 
response  to  these  stimuli  is  measured.  For  performance,  for  example,  the  external  stimuli  are 
events  arriving  at  the  system;  the  mechanisms  are  scheduling,  concurrency,  and  resource  man¬ 
agement;  and  the  measurements  are  latency  or  throughput.  For  modifiability,  for  example,  the 
external  stimuli  are  changes  to  the  system,  the  mechanisms  for  controlling  the  cost  of  changes 
are  encapsulation  and  data  indirection,  and  the  measurement  is  the  cost  of  a  collection  of 
changes. 

The  ATAM  consists  of  a  number  of  steps,  starting  with  a  presentation  of  the  business  drivers, 
the  architecture,  and  the  identification  of  architecture  styles  in  the  system.  This  is  followed  by 
the  generation  of  scenarios,  prioritization  of  the  scenarios,  and  mapping  the  prioritized  scenar¬ 
ios  onto  the  architecture  styles. 

The  ATAM  uses  stakeholder  perspectives  to  derive  a  collection  of  scenarios  giving  specific 
instances  for  usage,  performance  requirements  or  growth,  various  types  of  failures,  various 
possible  threats,  and  various  likely  modifications.  The  scenarios  are  used  for  the  evaluators  to 
understand  the  action  of  the  system  or  product  line.  The  evaluation  results  in  three  different 
types  of  output: 
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1 .  a  collection  of  “sensitivity”  or  “tradeoff”  points.  A  sensitivity  point  is  a  component  or 
decision  made  in  the  architecture  that  is  critical  for  the  achievement  of  a  particular  quality 
attribute.  A  tradeoff  point  is  a  sensitivity  point  that  is  sensitive  for  multiple  quality 
attributes 

2.  a  framework  for  reasoning  about  the  system  or  product  line.  The  framework  for  reasoning 
about  the  system  or  product  line  may  take  a  variety  of  forms.  It  may  be  the  discussion  that 
follows  from  the  exploration  of  a  scenario,  it  may  be  a  model  or  a  portion  of  a  model  and  a 
discussion  of  how  that  model  might  be  analyzed  when  instantiated,  or  it  may  be  a  formula 
that  represents  how  to  calculate  a  value  of  a  particular  quality  attribute. 

3.  a  list  of  issues,  or  decisions  not  yet  made.  The  list  of  issues  or  decisions  not  yet  made 
arises  from  the  stage  of  the  life  cycle  of  the  evaluation.  An  architecture  represents  a  collec¬ 
tion  of  decisions.  Not  all  decisions  may  have  been  made,  even  when  designing  the  archi¬ 
tecture.  Some  of  these  decisions  are  known  to  the  development  team  as  having  not  been 
made  and  are  on  a  list  for  further  consideration.  Others  are  unknown  to  the  development 
team  and  stakeholders. 

The  only  ATAM  step  affected  by  the  C4ISR1  Framework  products  is  the  scenario  generation. 

The  other  steps  in  the  evaluation  phase  are  not  affected  by  the  C4ISR,  since  they  assume  only 

that  some  architectural  representations  will  be  available  on  which  to  map  the  scenarios. 


1.  C4ISR  is  Command,  Control,  Communications,  Computers,  Intelligence,  Surveillance,  and  Reconnaissance. 
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2  C4ISR  Architecture  Framework 


The  C4ISR  Architecture  Framework  defines  three  views  (operational,  system,  and  technical) 
and  two  classes  of  products  (essential  and  supporting).  The  three  views  are  defined  in  the  Sec¬ 
tions  2.1.1  through  2.1.3  of  the  framework  document  and  are  summarized  below: 

1 .  The  operational  architecture  view  is  a  description  of  the  tasks  and  activities,  operational 
elements,  and  information  flows  required  to  accomplish  or  support  a  military  operation.  It 
contains  descriptions  (often  graphical)  of  the  operational  elements,  assigned  tasks  and 
activities,  and  information  flows  required  to  support  the  war-fighter.  It  defines  the  types  of 
information  exchanged,  the  frequency  of  exchange,  which  tasks  and  activities  are  sup¬ 
ported  by  the  information  exchanges,  and  the  nature  of  information  exchanges  in  detail 
sufficient  to  ascertain  specific  interoperability  requirements. 

2.  The  systems  architecture  view  is  a  description,  including  graphics,  of  systems  and  inter¬ 
connections  providing  for,  or  supporting,  war-fighting  functions.  For  a  domain,  the  sys¬ 
tems  architecture  view  shows  how  multiple  systems  link  and  interoperate,  and  it  may 
describe  the  internal  construction  and  operations  of  particular  systems  within  the  architec¬ 
ture.  For  the  individual  system,  the  systems  architecture  view  includes  the  physical  con¬ 
nection,  location,  and  identification  of  key  nodes  (including  materiel  item  nodes),  circuits, 
networks,  war-fighting  platforms,  etc.,  and  it  specifies  system  and  component  perfor¬ 
mance  parameters  (e.g.,  mean  time  between  failure,  maintainability,  availability).  The  sys¬ 
tems  architecture  view  associates  physical  resources  and  their  performance  attributes  to 
the  operational  view  and  its  requirements  per  standards  defined  in  the  technical  architec¬ 
ture. 

3.  The  technical  architecture  view  is  the  minimal  set  of  rules  governing  the  arrangement, 
interaction,  and  interdependence  of  system  parts  or  elements,  whose  purpose  is  to  ensure 
that  a  conforming  system  satisfies  a  specified  set  of  requirements.  The  technical  architec¬ 
ture  view  provides  the  technical  systems-implementation  guidelines  upon  which  engineer¬ 
ing  specifications  are  based,  common  building  blocks  are  established,  and  product  lines 
are  developed.  The  technical  architecture  view  includes  a  collection  of  the  technical  stan¬ 
dards,  conventions,  rules,  and  criteria  organized  into  profile(s)  that  govern  system  ser¬ 
vices,  interfaces,  and  relationships  for  particular  systems  architecture  views  and  that  relate 
to  particular  operational  views. 

The  framework  specifies  that  the  essential  products  must  be  supplied,  and  different  systems 
could  require  different  supporting  products  depending  on  the  context.  However,  it  does  not 
provide  more  specific  guidance  regarding  what  supporting  products  might  be  needed: 
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The  decision  of  which  products  to  build,  beyond  the  essential  set,  must  be  made 
based  on  the  issues  the  architecture  is  intended  to  explore  and  the  resulting 
characteristics  that  the  architecture  must  capture.  A  given  architecture  may 
contain  all  of  the  supporting  products,  a  selected  subset,  or  none  of  the  support- 
ing  products.  F or  example,  an  architecture  that  is  to  be  used  in  business  process 
reengineering  should  include  an  Activity  Model;  an  architecture  that  is  to  be 
used  in  examining  technology  insertion  and  achievable  states  of  “to-be”  capa¬ 
bilities  should  include  a  System  Technology  Forecast  [ C41SR  Architecture 
Framework  Version  2,  Section  3. 1.2.2]. 

The  situation  is  further  complicated  because  the  framework  does  not  provide  a  process  for 
generating  the  products.  Thus,  an  organization  developing  an  architecture  that  is  compliant 
with  the  C4ISR  Framework  could  be  faced  with  an  unbounded  amount  of  effort. 

The  Systems  Architectures  Laboratory  at  George  Mason  University  has  developed  a  C4ISR 
Architecture  Framework  Implementation  that  includes  a  process  for  generating  the  essential 
and  supporting  products  for  the  operational  and  systems  architecture  views.  The  process  takes 
a  systems  engineering  point  of  view  and  identifies  relationships  between  the  products  that  lead 
to  a  six-step  process: 

1.  STEP  0:  Problem  definition  and  collection  of  domain  information 

2.  STEP  1;  Operational  concept  and  requirements 

3.  STEP  2:  Functions  and  organizations 

4.  STEP  3:  Activity  model,  logical  data  model,  need  lines,  system  nodes,  system  elements 
and  functions,  and  task  allocation 

5.  STEP  4:  Operational  information  elements  and  exchanges,  system  functionality  descrip¬ 
tion,  physical  data  model 

6.  STEP  5:  System  information  elements  and  exchanges,  local  area  networks/wide  area  net¬ 
works  (LAN/WANs),  system  interface  descriptions,  system  performance 

This  process  is  shown  in  Figure  2-1,  which  was  reproduced  from  the  AFCEA1  Course  503E, 
Chapter  3,  Page  4. 


1 .  AFCEA  is  the  Armed  Forces  Communications  and  Electronics  Association. 
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The  process  is  complex  because  there  is  tight  coupling  among  products,  and  the  number  of 
relationships  increase  dramatically  with  the  layers  of  representation  in  the  activity  model  (OV- 
2)  and  the  system  functionality  description  (SV-4).  However,  the  process  produces  products  in 
all  six  steps,  allowing  for  a  continuous  review  and  evaluation  of  the  architecture  description 
process.1 

This  report  does  not  deal  with  product  generation;  we  assume  that  some  collection  of  products 
has  been  developed.  Instead,  we  focus  on  the  task  of  evaluating  the  architecture  described  by 
the  products.  The  preceding  discourse  simply  points  out  that  the  effort  required  to  generate  the 
products  is  not  trivial;  therefore,  it  is  important  to  choose  wisely.  As  we  shall  see,  depending 
on  the  combination  of  products  available  at  the  time  of  the  evaluation,  different  types  of 
attribute-specific  scenarios  can  be  generated.  Thus,  the  choice  of  products  could  be  driven  by 
the  quality  attributes  of  interest. 

One  reason  for  the  complexity  of  relationships  between  the  operational  and  system  views  is 
that  the  operational  activities  can  be  developed  quite  separately  from  the  system’s  functions. 
Hence  a  general  activity  can  involve  a  number  of  system  functions,  and  a  system  function  can 
perform  parts  of  many  activities,  leading  to  the  complex  relationships.  The  complexity  can  be 
reduced  by  carefully  choosing  the  activities  to  correspond  to  the  system  functions. 

The  techniques  used  in  the  C4ISR  Framework  derive  from  the  structured  analysis  tradition. 
Over  the  past  10  years,  object-oriented  methodologies  have  become  popular,  but  object  orien¬ 
tation  forces  the  developers  to  consider  only  relationships  between  objects  and  removes  the 
“activity”  versus  “function”  split.  The  Unified  Modeling  Language  (UML)  supports  the  devel¬ 
opment  of  object-oriented  views  and  is  becoming  an  industry  standard.  UML  products,  how¬ 
ever,  are  not  interchangeable  with  C4ISR  products  since  they  are  developed  with  a  different 
mindset  (structure  analysis  versus  object  orientation).  The  use  of  similar  names  for  different 
concepts  in  the  two  traditions  makes  it  difficult  to  know  what  is  meant  by  a  particular  term. 


1 .  Alexander  Levis  and  his  team  at  the  Systems  Architectures  Laboratory,  George  Mason  University  have  built  an  executable 
model  of  the  C4ISR  products.  Such  an  executable  model  could  be  used  to  validate  the  C4ISR  products,  by  developing,  ex- 
ecuting,  and  analyzing  many  test  cases. 
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3  Linking  the  C4ISR  Architecture  Frame¬ 
work  and  the  ATAM 


The  C4ISR  Framework  concentrates  on  view  and  product  definitions,  and  it  does  not  require 
use  cases  or  change  cases  to  clarify  and  assess  the  effectiveness  of  the  architecture.  The  ATAM 
Evaluation  phase  relies  on  generating  use-case  and  change-case  scenarios  to  analyze  the  archi¬ 
tectures.  It  is  not  immediately  obvious  which  types  of  use  cases  and  change  cases  can  or  can 
not  be  analyzed  against  which  architectural  views  and  products.  This  is  further  complicated  by 
the  fact  that  the  framework  allows  the  products  to  be  enhanced  by  annotations  or  by  innovative 
usage.  Products  may  even  be  replaced  by  alternative  products  that  are  deemed  to  be  equivalent 
to  the  C4ISR  Framework  products. 


3.1  C4ISR  Products  as  Potential  Sources  of 
Scenarios 

We  reviewed  the  descriptions  of  the  C4ISR  products  in  Section  4  of  the  framework  and  the 
product  attribute  tables  defined  in  Appendix  A  of  the  C4ISR  Architecture  Framework  Version 
2,  and  we  determined  how  could  they  be  used  to  generate  scenarios  for  use  cases  and  change 
cases  associated  with  performance,  dependability,  security,  modifiability,  and  interoperability. 

In  doing  so,  we  assumed  a  strict  interpretation  of  the  appendices.1  Our  detailed  observations 
are  listed  in  Appendix  A  and  are  summarized  in  Table  3-1  and  Table  3-2.  The  tables  indicates 
what  attribute-specific  information  is  contained  in  the  product,  information  which  could  be 
used  to  generate  attribute-specific  scenarios. 


1 .  Obviously  more  could  be  done  with  any  view  that  is  made  richer  by  annotations  or  innovation. 
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Possible  sources  of 
attribute-specific 
scenarios  from 
individual  products 

Performance 

Dependability 

Security 

Modifiability/ 

Interoperability 

Overview  and  Summary  Informa¬ 
tion  (AV-1) 

Integrated  Dictionary  (AV-2) 

High-Level  Operational  Concept 
Graphic  (OV-1) 

throughput 

latency 

exposure 

intersection 

spoofing 

Operational  Node  Connectivity 
Description  (OV-2) 

Operational  Information  Exchange 
Matrix  (OV-3) 

message  size 
media 

frequency  of 
exchanges 
timeliness 
throughput 

redundancy 

classification  of 
messages 

protocols 

formats 

LISI  levels* 

System  Interface  Description  (SV-1) 

throughput 

channel 

capacity 

latency 

redundancy  of 
components/ 
functions 

encryptions 

standards 

protocols 

components 

contained 

functions 

performed 

Technical  Architecture  Profile  (TV- 
1) 

protocols 

formats 

Table  3- 1:  Summary  of  Essential  Products 


a.  US1  is  levels  of  information  systems  interoperability. 
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alternative  paths 

throughput 

latency 


message  size 


WAN/LAN 
characteristics 
steps  in  path 


Command  Relationships  Chart  (OV- 
4) 


Activity  Model  (OV-5) 


Operational  Activity  Sequence  and 
Timing  Descriptions  (OV-6a,  6b, 
and  6c) 


Logical  Data  Model  (OV-7) 


Systems  Communications  Descrip¬ 
tion  (SV-2) 


Systems2  Matrix  (SV-3) 


Systems  Functionality  Description 
(SV-4) 


Operational  Activity  to  System 
Function  Traceability  Matrix  (SV-5) 


System  Information  Exchange 
Matrix  (SV-6) 


System  Performance  Parameters 
Matrix  (SV-7) 


System  Evolution  Description  (SV- 
B) 


System  Technology  Forecast  (SV-9) 


System  Activity  Sequence  and  Tim*  execution  paths 
ing  Descriptions  (SV-lOa,  10b,  and  event  times 
10c) 


Physical  Data  Model  (SV-1 1 )  message  size 


Standards  Technology  Forecast 
(TV-2) 


Table  3-2:  Summary  of  Supporting  Products 


frequency 

timeliness 

throughput 


data  rates 
throughput 
capacity 
initialization/ 
restart  time 
response  time 


backups 


flow  of 
activities 


points  of 
failure 


value  faults 


fault  rates 
redundancy 


classification 

encryption 

spoofing 


classification  functionality  evo¬ 
lution 


classification 

media 

format 

MTTFfor 

hardware/ 

software 


upgraded 

capabilities 


removing 
constraints  on 
functionality 
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3.2  C4ISR  Products  and  ATAM  Screening  Questions 

During  scenario  gathering,  the  evaluators  use  screening  questions  to  elicit  use-case  and 
change-case  scenarios  from  the  stakeholders,  and  they  prioritize  these  cases  to  determine 
which  ones  will  be  used  for  analysis.  The  high-priority  scenarios  will  be  a  mixture  of  scenar¬ 
ios,  some  that  are  strictly  operational  and  some  that  are  system  oriented. 

The  screening  questions  in  an  ATAM  are  targeted  at  what  is  equivalent  to  C4ISR  system-level 
views  rather  than  C4ISR  operational  views.  This  section  describes  how  these  questions  could 
be  extended  to  include  questions  targeted  at  the  C4ISR  operational  views.  Appendix  B  lists 
example  screening  questions  that  are  appropriate  for  operational  views,  and  Appendix  C  lists 
examples  of  actual  ATAM  scenarios. 

There  are  two  types  of  C4ISR  operational  view  screening  questions:  those  concerning  the 
operational  environment,  which  help  us  to  derive  an  operational  context  for  the  developing 
detailed  use-case  scenarios,  and  those  concerned  with  the  logistics  (support)  environment, 
which  help  us  to  derive  a  logistics  context  for  detailed  change-case  scenarios. 


3.2.1  Operational  Environment 

The  intent  of  the  screening  questions  (contained  in  Appendix  B.l)  is  to  move  from  a  very 
wide-scale  operational  problem  set  to  a  much  smaller  scale  problem  set  that  is  likely  to  expose 
the  architectural  weaknesses  and  strengths.  They  should  result  in  focusing  the  use  cases. 


3.2.1 .1  Mission  and  Geography 

These  questions  (contained  in  Appendix  B.1.1)  help  us  to  focus  on  those  issues  that  are  stress¬ 
ful  to  the  operational  architecture  and  those  that  are  difficult  to  accomplish.  Information 
uncovered  by  these  questions  can  also  create  a  general  background  environmental  “workload” 
that  represents  those  conditions.  A  workload  could,  for  example,  indicate  the  number  of  ves¬ 
sels  and  aircraft  within  the  area  of  interest  (AOI),  describe  how  an  operational  picture  is 
achieved  and  distributed  to  the  platforms  and  command  centers  involved  with  the  mission,  and 
show  the  acceptable  latency  for  upgrades  of  this  picture. 

3.2.1 .2  Organization  and  Command  Structure 

These  questions  (contained  in  Appendix  B.1.2)  help  us  to  establish  the  command  structures 
involved  and  how  the  structures  interact  to  conduct  the  missions.  These  questions  could  lead 
us  to  understand  which  missions  could  have  command-structure  interactions  that  are  problem¬ 
atic,  or  which  missions  will  have  command-structure  problems  when  failures  arise. 
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3.2.1 .3  Operational  Abstract  Quality  Attributes 

The  operational  environments  established  by  the  previous  screening  questions  can  now  be 
used  to  develop  abstract  scenarios  for  the  appropriate  quality  attributes,  such  as  availability, 
performance,  security,  and  interoperability.  For  each  such  quality  attribute,  there  are  seeding 
questions  (contained  in  Appendix  B.1.3)  that  will  lead  to  some  abstract  scenarios.  To  generate 
the  scenarios,  the  architect  will  have  to  reference  some  high-level  operational  views  of  the  sys¬ 
tem. 


3.2.2  Logistics  Environment 

The  screening  questions  (contained  in  Appendix  B.2)  will  help  us  focus  the  problems  with  sys¬ 
tem  upgrades  and/or  logistics.  They  should  lead  to  an  understanding  of  the  strategies  involved 
in  these  issues  and  allow  us  to  focus  the  change-case  scenarios  appropriately. 

3.2.2.1  Platform  and  Command  Center  Upgrades 

These  questions  (contained  in  Appendix  B.2.1)  help  us  to  establish  the  ground  rules  under 
which  platform  and/or  command-center  upgrades  will  be  made.  For  example,  how  are  classes 
of  cutters  and  aircraft  upgraded?  Are  cutters  upgraded  during  routine  maintenance  operations, 
or  can  certain  upgrades  be  done  less  intrusively?  Are  all  aircraft  of  a  certain  class  removed 
from  service,  their  crews  retrained,  and  then  all  restored  to  service?  The  screening  questions 
should  lead  to  a  basic  understanding  of  these  ground  rules  for  platform  and  command-center 
upgrades. 

A  related  issue  is  that  of  maintaining  platform  configurations.  How  much  variety  is  allowed 
between  platforms  in  the  same  and  different  operational  areas? 


3.2.3  Evolution  Strategy 

Many  of  the  infrastructure  components  within  certain  systems  become  obsolete  in  relatively 
short  time  periods.  There  can  be  a  strategy  of  extending  the  life  of  these  components  beyond 
its  normal  range  or  a  strategy  of  timely  replacement  of  components.  With  software,  the  prob¬ 
lem  is  compounded  due  to  the  relatively  short  lifetime  of  software  components.  These  ques¬ 
tions  (contained  in  Appendix  B.2.2)  will  help  us  to  focus  the  scenarios. 

3.2.4  Legacy  Systems 

Many  of  the  legacy  systems  will  remain  until  retirement,  and  in  some  cases  they  must  interop¬ 
erate  with  the  new  systems  being  introduced.  The  screening  questions  (contained  in  Appendix 
B.2.3)  in  this  area  should  lead  to  an  understanding  of  the  “ground  rules”  under  which  these 
operations  will  occur. 
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3.2.5  Logistics  Abstract  Quality  Attributes 

The  results  of  the  above  screening  process  should  allow  us  to  capture  the  major  issues  to  be 
considered  for  the  change  cases.  The  questions  (contained  in  Appendix  B.2.4)  will  help  us  to 
generate  change  cases  for  the  quality  attributes  of  modifiability  and  interoperability.  The  pro¬ 
cess  here  parallels  that  of  the  use  cases,  concerning  levels  of  abstraction  and  detail  in  both  the 
change-case  scenarios  and  the  view  that  are  referenced. 
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4  Conclusions  and  Recommendations 


The  product  summaries  in  Table  3-1  and  Table  3-2  suggest  how  each  product  could  contribute 
to  the  development  of  quality  attribute-specific  scenarios  by  using  the  ATAM  screening  ques¬ 
tions. 

The  products  are  complementary  with  respect  to  quality  attributes,  and  different  combinations 
of  products  could  be  used  to  generate  different  scenarios,  depending  on  the  attributes  of  inter¬ 
est.  For  example.  Table  4-1  show  the  consolidation  of  all  the  essential  products,  the  supporting 
operational  view  products,  and  selected  products  focused  on  system  evolution.  Any  number  of 
these  combinations  could  be  used  as  guidance  to  identify  products  that  are  desirable  for  gener¬ 
ating  attribute-specific  scenarios. 


Possible  sources  of  attribute- 
specific  scenarios  from  product 
combinations 

Performance 

Dependability 

Security 

Modifiability/ 

Interoperability 

Essential  Products 

•  Overview  and  Summary  Information  (AV-1) 

•  Integrated  Dictionary  (AV-2) 

•  High-Level  Operational  Concept  Graphic  (OV- 1 ) 

•  Operational  Node  Connectivity  Description  (OV-2) 

•  Operational  Information  Exchange  Matrix  (OV-3) 

•  System  Interface  Description  (SV-1) 

•  Technical  Architecture  Profile  (TV-1) 

channel 

capacity 

message 

size 

media 

frequency  of 
exchanges 
timeliness 
throughput 
latency 

redundancy  of 
components/ 
functions 
points  of 
failure 
availability 
of  assets 

classification 
of  messages 
encryptions 
exposure 
intersection 
spoofing 

standards 

protocols 

formats 

components 

contained 

functions 

performed 

formats 

LISI  levels 

Supporting  Operational  View  Products 

•  Command  Relationships  Chart  (OV-4) 

•  Activity  Model  (OV-5) 

•  Operational  Activity  Sequence  and  Timing  Descrip¬ 
tions  (OV-6a,  6b,  and  6c) 

•  Logical  Data  Model  (OV-7) 

alternative 

paths 

throughput 
latency 
message  size 

backups 
flow  of 
activities 
points  of 
failure 
value  faults 

Supporting  Products  Focused  on  System  Evolution 

•  System  Evolution  Description  (SV-8) 

•  System  Technology  Forecast  (SV-9) 

•  Standards  Technology  Forecast  (TV-2) 

! 

upgraded 

functionality 

upgraded 

capabilities 

new 

standards 

Table  4-1;  Product  Combinations 


CMU/SEI-99-TR-01 4 


13 


Both  use-case  and  change-case  scenarios  can  be  expressed  at  the  level  consistent  with  the 
C4ISR  operational  views  or  system  views.  Scenarios  expressed  at  the  level  of  the  operational 
views  can  be  further  refined  in  terms  of  the  more  detailed  system  views,  as  described  below: 

•  Scenarios  at  the  level  of  operational  views  will  be  expressed  in  terms  of  nodes,  informa¬ 
tion  flows,  and  activities.  These  can  be  analyzed  against  the  operational  views. 

•  Scenarios  at  the  level  of  the  system  views  will  be  expressed  in  terms  of  systems,  functions, 
and  communication  paths.  These  can  be  analyzed  against  the  system  views. 

For  example,  the  following  scenarios  could  be  analyzed  against  operational  views: 

•  A  use-case  scenario  may  explore  the  impact  of  a  node  failure  on  a  specific  mission  in  a 
specific  geographical  region. 

•  A  change-case  scenario  might  explore  the  impact  of  adding  a  new  communications  satel¬ 
lite  system  to  a  mission. 

If  the  C4ISR  operational  view  products  are  sufficiently  detailed,  the  scenarios  can  be  analyzed 
to  determine  their  impact  on  the  mission  in  terms  of  the  nodes,  command  structure,  activities, 
and  communication  paths.  However,  the  scenarios’  impact  on  specific  systems  or  functions 
cannot  be  determined  at  this  level.  In  these  cases,  the  scenarios  must  be  refined  in  terms  of  sys¬ 
tems  and  functions  and  their  connectivity,  and  then  explored  against  the  C4ISR  system  views. 
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Appendix  A:  C4ISR  Products  as  Sources 

of  Scenarios 


The  C4ISR  Framework  distinguishes  between  essential  and  supporting  products,  and  for  each 
product  it  defines  the  information  to  be  included  in  the  product.  These  information  tables  are 
captured  in  the  integrated  dictionary,  which  is  itself  one  of  the  products  (AV-2). 

In  this  appendix,  we  identify  items  in  the  information  tables  that  could  be  used  to  derive  sce¬ 
narios.  This  is  a  literal  scan  of  the  product  information  tables,  looking  for  implicit  or  explicit 
references  to  quality-attribute  concerns.  Further,  we  suggest  how  combining  the  views  could 
provide  coverage  across  multiple  attributes. 


A.1  Essential  Products 

A.1.1  Overview  and  Summary  Information  (AV-1) 

This  view  is  both  a  planning  guide  and  documentation  upon  completion  of  the  development. 

It  can  be  used  to  identify  assumptions,  constraints,  attribute-specific  requirements,  and  scenar¬ 
ios. 

A.1 .2  Integrated  Dictionary  (AV-2) 

This  is  a  central  source  for  definitions  and  annotations  to  the  graphical  representations. 

It  can  be  used  to  document 

•  attribute-specific  data  (e.g.,  rates,  probabilities,  impact  of  changes) 

•  risks  identified  during  the  architecture  evaluation 


CMU/SEI-99-TR-01 4 


15 


A.1.3  High-Level  Operational  Concept  Graphic  (OV-1) 

A  graphical  representation  of  operations  in  terms  of  such  things  as  missions,  functions,  organi¬ 
zations,  and/or  asset  distribution.  It  describes  weapon  types,  logical  and  physical  connections, 
trajectories  and  targets,  and  criticality  of  mission. 

It  can  be  used  to  assign  weights  to  attribute-specific  evaluations,  sensitivities,  and  tradeoffs. 
Possible  scenarios  include 

•  dependability  (from  points  of  failure  and  availability  of  assets) 

•  performance  (from  throughput  and  latency  of  communications) 

•  security  (from  exposure,  interception,  and  spoofing) 


A.1 .4  Operational  Node  Connectivity  Description  (OV-2) 

This  describes  the  operational  nodes,  the  need  lines  between  them,  and  the  characteristics  of 
the  information  exchanged.  Details  appear  in  OV-3. 


A.1. 5  Operational  Information  Exchange  Matrix  (OV-3) 

This  matrix  captures  requirements  for  information  exchanges  between  operational  nodes.  It 
adds  details  to  OV-2. 

Possible  scenarios  include 

•  interoperability  (from  protocols,  formats  and  LISI  levels) 

•  security  (from  classification  of  messages) 

•  dependability  (from  redundancy) 

•  performance  (from  message  size,  media,  frequency  of  exchange,  timeliness,  and  through¬ 
put  data) 


A.1 .6  System  Interface  Description  (SV-1) 

This  description  links  together  the  operational  and  systems  architecture  views  by  depicting  the 
assignments  of  specific  systems  and  their  interfaces  to  the  nodes  and  need  lines  described  in 
OV-2.  See  SV-2  for  communication  nodes.  See  SV-6  for  system  information  elements 

Possible  scenarios  include 

•  interoperability  (from  standards  and  protocols) 

•  modifiability  (from  components  contained  and  functions  performed  by  each  component) 
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•  dependability  (from  redundancy  of  components/functions) 

•  performance  (from  throughput,  channel  capacity,  and  latency  data) 

•  security  (from  encryption) 

A.1.7  Technical  Architecture  Profile  (TV-1) 

This  profile  provides  a  time-phased  enumeration  of  the  relevant  subset  of  technical  standards 
that  apply  to  the  architecture  and  how  they  have  been  or  are  to  be  implemented. 

Possible  scenarios  include 

•  interoperability  (from  protocols) 

•  modifiability  (from  protocols  and  formats) 


A.2  Supporting  Products 

A.2.1  Command  Relationships  Chart  (OV-4) 

This  chart  illustrates  the  hierarchy  of  organizations  or  resources  in  an  architecture  and  the  rela¬ 
tionship  among  diem  (e.g.,  command,  control,  coordination). 

A  possible  scenario  is  dependability  (from  backups). 


A.2.2  Activity  Model  (OV-5) 

This  model  describes  the  applicable  activities  associated  with  the  architecture,  the  data  and/or 
information  exchanged  between  activities,  and  the  data  and/or  information  exchanged  with 
other  activities  outside  the  scope  of  the  model  (i.e.,  external  interfaces).  See  OV-2  for  opera¬ 
tional  node  and  OV-3  for  operational  information  element. 

A  possible  scenario  is  dependability  (from  flow  of  activities). 


A.2.3  Operational  Activity  Sequence  and  Timing  Descriptions 
(OV-6a,  6b,  and  6c) 

These  descriptions  consist  of  a  set  of  three  types  of  models  needed  to  refine  and  extend  the 
operational  view,  to  adequately  describe  the  dynamic  behavior  and  performance  characteris¬ 
tics  of  the  business  processes  critical  to  an  architecture.  See  OV-2  for  operational  nodes.  These 
three  models  are  described  below: 
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•  Operational  rules  model  (OV-6a)  —  operational  rules  expressed  in  a  formal  language 

•  Operational  state  transition  description  (OV-6b)  —  detailed  timing  sequence  of  activities 
or  work  flow  in  the  business  process 

•  Operational  event/trace  description  (OV-6c)  —  depiction  of  the  dynamic  behavior  of  mis¬ 
sion  processes  (used  alone  or  in  conjunction  with  OV-6b) 

Possible  scenarios  include 

•  performance  (from  alternative  paths,  throughput,  and  latency) 

•  dependability  (from  points  of  failure) 


A.2.4  Logical  Data  Model  (OV-7) 

This  model  describes  the  data  and  information  that  are  associated  with  the  information 
exchanges  of  the  architecture. 

Possible  scenarios  include 

•  performance  (from  message  size) 

•  dependability  (from  value  faults) 


A.2.5  Systems  Communications  Description  (SV-2) 

This  description  represents  the  specific  pathways  or  networks  of  the  communications  systems 
and  the  details  of  their  configurations  through  which  the  physical  node  and  systems  interface. 
See  SV-1  for  systems  nodes,  systems,  and  links.  See  OV-2  for  need  lines  and  operational 
nodes. 

Possible  scenarios  include 

•  performance  (from  WAN/LAN  characteristics  and  steps  in  the  path) 

•  dependability  (from  fault  rates  and  redundancy) 

•  security  (from  classification,  encryption  and  spoofing) 


A.2.6  Systems2  Matrix  (SV-3) 

This  is  a  description  of  the  system-to-system  relationships  identified  in  the  various  types  (e.g., 
intemodal  and  intranodal)  in  SV-1.  See  SV-6  for  system  information  elements. 
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Possible  scenarios  include 


•  interoperability  (from  existing/planned/potential/deactivated  functionality) 

•  security  (from  classification  of  data) 


A.2.7  Systems  Functionality  Description  (SV-4) 

This  describes  the  flow  of  data  among  system  functions,  and  the  relationship  between  systems 
or  system  functions  and  activities  at  nodes.  See  SV-1  for  system  function  and  system  node.  See 
SV-6  for  system  information  element. 

This  is  not  a  likely  source  of  scenarios. 


A.2.8  Operational  Activity  to  System  Function  Traceability 
Matrix  (SV-5) 

This  matrix  helps  to  link  the  operational  and  systems  architecture  views  by  depicting  the 
“many-to-many”  mappings  of  operational  activities  to  systems  functions.  See  SV-1  for  system 
function.  See  OV-5  for  operational  activity. 

This  is  not  a  likely  source  of  scenarios. 


A.2.9  System  Information  Exchange  Matrix  (SV-6) 

This  matrix  describes,  in  tabular  form,  the  physical  aspects  of  how  the  information  exchanges 
called  for  in  OV-2  actually  are  (or  will  be)  implemented  in  terms  of  protocols,  data  formats, 
etc. 

This  is  particularly  useful  for  understanding  the  potential  for  overhead  and  constraints  intro¬ 
duced  by  these  choices.  See  SV1  for  system,  system  elements,  system  components,  system 
functions.  See  SV-4  for  the  inputs  to  and  outputs  from  the  system  elements. 

Possible  scenarios  include 

•  interoperability  (from  media  and  format) 

•  security  (from  classification) 

•  performance  (from  frequency,  timeliness,  and  throughput  data) 
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A.2.10  System  Performance  Parameters  Matrix  (SV-7) 

This  matrix  builds  on  the  “system  element  interface  description”  (SV-1)  by  portraying  the  cur¬ 
rent  hardware  and  software  performance  characteristics  of  each  system,  and  the  or  required 

performance  characteristics  at  specified  times  in  the  future,  geared  towards  TV-2.  See  SV-1  for 

system,  system  element,  and  system  component. 

This  product  provides  the  following  data  that  are  needed  for  attribute-specific  analyses: 

•  mean  time  to  failure  (MTTF),  maintainability,  availability,  system  initialization  time,  data 
transfer  rate,  and  program  restart  time  for  platforms 

•  data  throughput/capacity,  response  time,  effectiveness,  and  mean  time  between  software 
failures  for  application  software 

Possible  scenarios  include 

•  dependability  (from  MTTF  for  hardware  and  application  software) 

•  performance  (from  data  transfer  rate  and  throughput/capacity,  system  initialization,  pro¬ 
gram  restart,  and  response  time) 


A.2.11  System  Evolution  Description  (SV-8) 

This  depicts  how  a  suite  of  systems  will  be  made  more  modem  over  time,  including  evolution 
and/or  migration  steps  to  accommodate  the  specific  information  requirements,  performance 
parameters,  and  technology  forecasts  provided  in  other  products.  See  SV-1  for  system,  system 
element,  and  system  component. 

A  possible  scenario  is  modifiability  (from  upgraded  functionality). 


A.2.12  System  Technology  Forecast  (SV-9) 

This  contains  predictions  about  the  availability  of  emerging  technologies,  specific  hardware/ 
software  products,  and  industry  trends  in  short-,  mid-,  and  long-term  intervals,  focused  on 
technology  areas  that  are  relevant  to  the  architecture’s  purpose. 

A  possible  scenario  is  modifiability/interoperability  (from  upgraded  capabilities). 
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A.2.13  System  Activity  Sequence  and  Timing  Descriptions 
(SV-1  Oa,  1 0b,  and  1 0c) 

These  descriptions  consist  of  three  types  of  models  needed  to  refine  and  extend  the  systems 
view,  to  adequately  describe  the  dynamic  behavior  and  performance  characteristics  of  the 
architecture.  The  three  descriptions  are  described  below: 

•  Systems  rules  model  (SV-lOa)  —  constraints  imposed  on  system  functionality  due  to 
some  aspect  of  system  design  or  implementation,  expressed  in  a  formal  language 

•  Systems  state  transition  description  (SV-lOb)  —  detailed  sequencing  of  functions  in  a  sys¬ 
tem 

•  Systems  event/trace  description  (SV-lOc)  —  description  of  dynamic  behavior,  tracing  the 
actions  in  a  scenario  or  critical  sequence  of  events  along  a  given  time  line  (used  alone  or  in 
conjunction  with  SV-lOb) 

This  product  may  reflect  system-specific  aspects  or  refinements  of  critical  sequences  of  events 
described  in  the  operational  view  (e.g.,  performance-critical  scenarios).  See  OV-6a,  6b,  and  6c 
for  the  sequences  of  events. 

Possible  scenarios  include 

•  modifiability  (from  removing  constraints  on  functionality) 

•  performance  (from  execution  paths  and  event  times) 

•  dependability  (from  points  of  failure) 

•  security  (from  points  of  attack) 


A.2.14  Physical  Data  Model  (SV-11) 

Describes  how  the  information  represented  in  OV-7  is  actually  implemented  (that  is,  how  the 
information  exchange  requirements  actually  are  implemented  and  how  both  data  entities  and 
their  relationships  are  maintained  in  the  systems  architecture). 

Possible  scenarios  include 

•  performance  (from  message  size) 

•  dependability  (from  file  system  properties) 


A.2.15  Standards  Technology  Forecast  (TV-2) 

This  consists  of  detailed  descriptions  of  emerging  technology  standards  and  implementing 
products  that  are  relevant  to  the  systems  and  business  processes  covered  by  the  architecture  in 
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short-,  medium-,  and  long-term  intervals,  with  confidence  factors  for  the  prediction,  along 
with  issues  that  may  affect  the  architecture.  See  TV-1  and  SV-9  for  the  timeline  of  the  stan¬ 
dards  and  the  prediction  of  availability  of  the  emerging  technologies. 

A  possible  scenario  is  modifiability/interoperability  (from  new  standards). 


A.3  Relationships  Between  C4ISR  Products 

[Source:  product  attribute  tables  in  Appendix  A  of  the  C4ISR  Framework.] 


Product 

Gets  details  from 

Implied  entities,  attributes  & 
relationships 

Overview  and  Summary 
Information  (AV-1) 

OV-l 

•  Organization 

Integrated  Dictionary  (AV- 
2) 

High-Level  Operational 
Concept  Graphic  (OV-1) 

OV-3 

•  Operational  Information 
Element 

Operational  Node  Con¬ 
nectivity  Description  (OV- 
2) 

OV-3 

•  Operational  Information  Element 
OV-5 

•  Activity 

Operational  Information 
Exchange  Matrix  (OV-3) 

OV-2 

•  Need  line 

•  Operational  Node 

OV-5 

•  Activity 

System  Interface  Descrip¬ 
tion  (SV-1) 

SV-2 

•  Communication  Node 

OV-2 

•  Need  line 

Technical  Architecture 
Profile  (TV-1) 

Table  A-1:  Detailed  Observations  About  Essential  Products 
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Product 


Gets  details  from 


Implied  entities,  attributes  & 
relationships 


Command  Relationships  OV-1 

Chart  (OV-4)  •  Organization 


Activity  Model  (OV-5)  OV-2 


Operational  Activity 
Sequence  and  Timing 
Descriptions  (OV-6a,  6b, 
and  6c) 


LogicalDataModel  (OV-7) 


Systems  Communications  SV-1 
Description  (SV-2)  .  Systems  Node 

•  System 


Need  line 
Operational  Node 


Systems  Matrix  (SV-3)  SV-l 


System 


System  Information  Element 


Systems  Functionality  S  V- 1 

Description  (SV-4)  .  Systems  Function 

•  Systems  Node 


Operational  Activity  to  SV-1 

System  Function  Trace-  .  Systcm  Function 

ability  Matrix  (SV-5)  - 


Systems  node  contains  system 
Operational  node  maps  to  systems  node 
Link  implements  need  line 


System  Information  Element 


•  Operational  Activity 

•  System  Function 


Table  A-2:  Detailed  Observations  About  Supporting  Products 
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Product 


Gets  details  from 


Implied  entities,  attributes  & 
relationships 


System  Information 
Exchange  Matrix  (SV-6) 


System  Performance 
Parameters  Matrix  (SV-7) 


System  Evolution 
Description  (SV-8) 


System  Technology  Fore¬ 
cast  (SV-9) 


System  Activity  Sequence 
and  Timing  Descriptions 
(SV-lOa,  10b,  and  10c) 


Physical  Data  Model  (SV- 
11) 


Standards  Technology 
Forecast  (TV-2) 


System 

System  Element 
System  Component 


System,  System  Element 

Application  Software  (System  Compo¬ 
nent) 

System  Function 

System  performs  system  function 

System  element  performs  system  func¬ 
tion 

System  information  element  is  input  to 
system  function 

System  is  source  of  system  information 
element 


System,  System  Element 

Application  Software  (System  Compo¬ 
nent) 

System  contains  system  element 
System  element  contains  system  compo- 


Reference  model  includes  service  area 
Service  area  includes  service 


Table  A-2:  Detailed  Observations  About  Supporting  Products  (Continued) 
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Appendix  B:  Screening  Questions  for  the 

C4ISR  Operational  View 


B.1  Operational  Environment 

B.1.1  Mission  and  Geography 

•  Which  missions  in  which  geographical  regions  would  you  expect  to  place  stress  on  the 
system  resources? 

•  How  would  you  define  the  workload  of  each  such  stressful  situation? 

•  What  assets  would  you  expect  to  be  deployed? 

When  these  have  been  answered  and  follow-up  questioning  and  explanations  have  been  com¬ 
pleted,  we  should  have  an  operational  picture  of  the  expected  stressful  situation.  Presumably 
the  architect  would  use  the  operational  views  to  represent  this  picture.  This  will  help  us  to 
understand  what  is  going  on. 


B.1 .2  Organization  and  Command  Structure 

•  What  is  the  command  structure  for  the  mission/geographical  region? 

•  How  does  this  command  structure  interweave  with  the  U.S.  Coast  Guard  (USCG)  opera¬ 
tional  hierarchy? 

•  Can  this  structure  change  with  asset  or  communications  failure? 


B.1 .3  Quality  Attributes 

B.1 .3.1  Dependability 

•  When  the  assets  are  deployed  on  a  mission,  are  there  different  ways  in  which  the  commu¬ 
nications  system  can  fail  (for  example,  going  out  of  line  of  site,  equipment  failure)? 

•  Are  their  redundant  communication  paths,  and  how  is  reconfiguration  achieved  on  failure? 

•  Are  their  particular  assets  whose  failure  would  cause  the  mission  to  be  degraded  or  aban¬ 
doned? 


CMU/SEI-99-TR-014 


25 


What  types  of  fixes  to  failed  equipment  are  done  during  the  mission?  Are  there  trained 
personnel  and  spare  parts  on  board? 

Are  there  cases  where  faults  can  propagate  causing  widespread  failures? 


B.1 .3.2  Performance 

Bandwidth 

•  How  much  information  will  be  transmitted  over  the  communications  paths  and  at  what 
intervals? 

•  Are  there  bursts  of  information  that  could  saturate  the  communications  paths  and  cause 
delays  to  critical  information? 

•  Are  there  large  messages  being  transmitted,  which  could  interfere  with  other  traffic  (e.g., 
initialization,  charts)? 

Response  Times 

•  Do  operational  personnel  require  some  critical  responses,  and  what  type  of  delay  is 
acceptable?  Have  you  performed  an  analysis  to  ensure  that  this  can  be  met  under  high- 
stress  burst  conditions  within  reason? 

•  Are  there  some  critical  response  times  to  detected  trigger  events? 


B.1. 3.3  Security 

•  Are  there  missions  where  the  enemy  will  be  attempting  to  create  false  alarms  to  confuse 
the  mission?  How  can  you  prevent  these  false  alarms  from  affecting  the  mission? 

•  Is  there  critical  information  that  must  be  consistent  and  have  protected  access  against 
unauthorized  disclosure? 

•  Is  multi-level  access  necessary? 

•  Is  denial  of  service  an  issue? 

B.2  Logistics  Environment 

B.2.1  Platform  and  Command  Center  Upgrades 

•  Which  anticipated  platform  upgrades  will  be  most  challenging  for  team  formation  and 
mission  operation? 

•  Which  asset  upgrades  will  happen  together  (all  operational  assets  of  Type  A  will  be 
removed  from  service,  upgraded,  and  restored  to  service  together),  and  which  will  be  done 
on  a  one-at-a-time  basis? 
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•  Will  different  types  of  upgrades  have  different  strategies  (e.g.,  new  functionality,  technol¬ 
ogy  refreshment,  problem  fixes)? 

B.2.2  Evolution  Strategy 

•  Is  there  a  policy  about  keeping  technology  relatively  fresh? 

•  Will  components  be  sustained  beyond  their  end  of  life? 

•  Is  there  a  policy  about  installing  obsolete  technology? 

B.2.3  Legacy  Systems 

•  Are  legacy  systems  to  be  upgraded  as  part  of  the  plan? 

•  How  will  the  legacy  system  upgrades  be  accomplished? 

B.2.4  Quality  Attributes 

B.2.4.1  Modifiability 

•  Is  there  a  list  of  expected  changes  that  will  occur  to  the  system  during  its  life  expectancy 
(e.g.,  threats,  technology,  operational,  budget,  schedule)? 

•  How  have  these  predicted  changes  affected  the  system  architecture? 

•  Have  contingency  plans  been  developed  based  on  these  predicted  changes? 

•  Are  the  requirements  likely  to  remain  stable  or  to  change  intrusively? 

B.2.4.2  Inter-operability 

•  Are  there  well-defined  procedures  for  developing  interfaces  to  external  agencies? 

•  Are  all  such  interfaces  defined  by  some  standard? 

•  Is  there  a  plan  to  configuration  manage  the  interfaces? 

•  Is  there  a  common  data  model  for  interfaces  shared  by  all  participating  agencies? 

The  above  questions  could  also  be  important  for  different  integrated  deep-water  system  (IDS) 
assets. 
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Appendix  C:  Example  ATAM  Scenarios 


The  following  example  scenarios  were  generated  during  actual  ATAM  evaluations. 


C.1  Scenarios  About  the  Development  Project 

Scenario  [Stakeholder (s);  Quality/Qualities] 

Decrease  cost  of  development  by  20%.  [Customer,  project  manager;  cost,  ability  to  subset] 

Project  manager  adds  two  new  developers  in  a  way  to  make  them  productive  quickly.  [Devel¬ 
oper,  project  manager;  buildability,  conceptual  integrity] 

Implement  a  skeletal  version  of  the  system  first.  Add  functionality  incrementally.  [Project 
manager,  developers,  integrators,  testers;  ability  to  subset] 

An  error  is  discovered  during  testing.  The  development  organization  identifies  the  source  of 
the  error  and  corrects  it.  [Developer,  maintainer;  maintainability,  testability] 

Schedule  the  threads  and  processes;  verify  that  the  set  is  schedulable.  [Developer,  performance 
engineer,  performance,  schedulability] 

The  network  is  overloaded  (for  network-based  systems).  Re-partition  tasks  to  reduce  traffic. 
[Developer,  performance  engineer;  performance,  schedulability] 

Predict  priority,  execution  time,  memory  space  usage,  deadline  satisfaction,  and  resource  con¬ 
tention  when  adding  new  software.  [Developer,  performance  engineer;  schedulability,  concep¬ 
tual  integrity] 

Produce  a  version  of  the  system  that  runs  on  a  desktop  machine  (for  systems  that  run  on  spe¬ 
cialized  platforms  or  are  embedded).  [Developers;  portability] 
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C.2  Scenarios  About  the  System  During  Operation 

Scenario  [Stakeholder(s);  Quality/Qualities] 


One  of  the  outputs  is  erratic.  Track  down  and  fix  the  bug.  [Tester,  maintainer;  testability,  mod¬ 
ifiability] 

An  error  (e.g.,  divide  by  zero)  occurs  during  processing.  How  does  the  system  detect  it,  report 
it,  handle  it,  and/or  correct  it?  [User,  tester,  developer,  maintainer,  field  support;  conceptual 
integrity,  testability] 

System  initialization  and  start-up  occurs.  How  does  the  architecture  handle  it?  Perhaps  appli¬ 
cation  code  must  run  before  system  start-up  is  complete.  [Developer,  user,  maintainer;  concep¬ 
tual  integrity,  modifiability] 

Allow  a  user  to  access  the  system  remotely  (e.g.,  via  the  Web,  via  radio  signals,  via  dial-up). 
[User;  modifiability,  security] 

Verify  the  system’s  computations,  and/or  the  timing  of  those  computations,  at  runtime  [Tester, 
maintainer,  performance  engineer;  modifiability,  fault  tolerance,  reliability] 

One  processor  or  server  goes  down  during  operation.  Half  of  the  processors  or  servers  go 
down  during  operation.  The  network  fails  during  operation.  One  or  more  input  devices  fail 
during  operation.  [User;  fault  tolerance,  availability] 

Transmit  data  (to  other  systems,  or  across  the  network  in  this  system)  in  encrypted  form. 

[User;  security] 

If  using  a  distributed  system,  change  the  processor  on  which  a  specific  piece  of  software  runs. 
[User,  maintainer,  performance  engineer;  fault  tolerance,  scalability,  portability] 

If  using  a  distributed  system,  use  a  fault  tolerance  scheme  that  accounts  for  software  migration 
off  of  a  failed  or  damaged  processor.  [Maintainer;  modifiability,  fault  tolerance,  availability] 

Degraded  operation  mode:  A  user  wants  to  continue  to  operate  the  system  (perform  useful 
work)  in  the  presence  of  one  of  the  following  kinds  of  failures:  database  server,  application 
server,  or  partial  network  failure.  [User,  maintainer;  fault  tolerance,  availability,  modifiability] 

Reduce  system  start-up  (or  re-start)  time  by  a  given  factor.  [User,  performance  engineer;  mod¬ 
ifiability,  performance] 
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An  algorithm  fails;  system  must  revert  back  to  using  a  previous  (and  more  reliable)  version. 
[User;  fault  tolerance] 

Only  certain  users  can  access  certain  data  in  the  database.  [User;  modifiability,  security] 
Have  the  system  operate  for  24  hours  a  day,  7  days  a  week.  [User;  reliability] 


C.3  Scenarios  About  Maintaining  or  Upgrading  the 
System 

Scenario  [Stakeholder(s);  Quality/Qualities] 

Add  a  new  data  server.  [Maintained  modifiability,  performance] 

Replace  one  of  the  input  devices  with  one  of  similar  or  slightly  enhanced  capability.  [Main¬ 
tained  modifiability] 

A  new  input  device  replaces  five  current  ones.  [Maintained  modifiability] 

Performance  requirements  are  doubled.  Data  throughput  is  doubled.  [Maintained  performance 
engineer;  performance,  modifiability] 

The  number  of  users  and/or  the  number  of  inputs  doubles.  [Maintained  performance  engineer; 
performance,  modifiability,  scalability] 

The  requirement  to  use  a  particular  commercial  off-the-shelf  (COTS)  component  is  levied. 
[Maintained  openness,  modifiability] 

The  computer  is  replaced.  If  there  is  more  than  one  computer,  heterogeneous  computers  are 
introduced.  The  compiler  is  replaced.  The  operating  system  is  replaced.  The  network  or  bus  is 
replaced.  [Maintained  modifiability,  portability] 

Add  a  completely  new  function;  compute  a  new  result.  [Customer,  user;  modifiability,  concep¬ 
tual  integrity] 

Make  the  system  interact  with  some  other  (specified)  system  in  some  (specified)  way.  (For 
example,  have  the  system  participate  in  a  cross-system  simulation  exercise.)  [User,  users  of 
other  system;  modifiability] 

Introduce  a  new  data  format  or  message  format.  [User,  users  of  other  system;  modifiability] 


CMU/SEI-99-TR-01 4 


31 


Introduce  a  recording/playback  capability.  [User;  modifiability] 

Replace  the  middleware  or  communications  infrastructure  with  one  supplied  by  a  different 
vendor.  [Customer,  performance  engineer;  modifiability] 

Add  automated  decision  aids.  [User;  modifiability] 

Replace  the  database.  Add  an  object-oriented  database.  [Developer,  customer;  modifiability] 
Replace  the  user  interface.  [User,  developer;  modifiability] 

Introduce  data  from  a  completely  new  source;  introduce  real-time  data.  [User,  customer;  mod¬ 
ifiability,  performance] 


C.4  Scenarios  Specifically  About  a  Product  Line 
Architecture 

Scenario  [Stakeholder(s);  Quality/Qualities] 


A  product  developer  wants  to  build  the  smallest  possible  running  system  (using  only  the  archi¬ 
tecture  and  some  subset  of  the  reusable  assets  of  the  product  line)  that  computes  some  useful 
result.  [Product  line  application  developer;  applicability  of  architecture  to  product  line,  con¬ 
ceptual  integrity,  ability  to  subset] 

A  new  reuser  (subscriber)  wishes  to  join  the  product  line,  but  needs  to  know  if  his  performance 
and  accuracy  requirements  can  be  met  using  the  core  assets.  [Reuser,  (new)  product  line  appli¬ 
cation  developer;  applicability  of  architecture  to  product  line,  conceptual  integrity,  ability  to 
subset] 

Issue  a  new  release  of  the  core  asset  library.  [Product  line  application  developers;  applicability 
of  architecture  to  product  line] 

An  error  is  found  in  a  member  of  the  product  line,  in  the  field.  How  is  the  error  corrected,  and 
how  does  it  affect  the  core  assets  and  other  members  of  the  product  line?  [Testers,  service  rep¬ 
resentatives;  ability  of  architecture  to  isolate  errors] 

The  domain  (scope)  of  the  product  line  is  expanded  to  include. . . .  [Product  line  manager; 
marketers;  product  line  application  developers;  flexibility,  applicability  of  architecture  to 
scope,  modifiability] 
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